1. Technical Field
The invention is related to blending or mixing of two or more songs, and in particular, to a system and process for automatically blending different pieces of music of arbitrary genres, such as, for example, automatically blending a heavily beat oriented song (i.e., a “Techno” type song) with a melodic song, such as a piano tune by Mozart, using automatic time-scaling, resampling and time-shifting without the need to determine beats-per-minute (BPM) of the blended songs.
2. Related Art
Conventional music mixing typically involves the blending of part or all of two or more songs. For example, mixing may involve blending the end of Song A into the beginning of Song B for smoothly transitioning between the two songs. Further, such mixing may also involve actually combining Song A and Song B for simultaneous playback to create a mixed song comprised of both Song A and Song B.
Clearly, simply playing two songs at the same time without any intervention would typically result in a discordant mix of unaligned music. Therefore, successful music mixing typically involves a number of factors that must be considered on a song-by-song basis. For example, these factors often include determining which song to transition into from a current song; when to do the transition in Song A; where in Song B to cut into; any timescale adjustment necessary to align Song A to Song B; and any time offsets required to align Song A and Song B.
There are a number of conventional schemes which are used for automatically mixing or blending two or more songs. Such schemes are frequently used by “DJ's” for mixing two or more songs to provide mixed dance music in real time, and to transition from one song to another as smoothly as possible. These conventional schemes include a variety of software and hardware tools, and combinations of both software and hardware.
In general, these conventional schemes typically operate by first estimating a “beats-per-minute” (BPM) count of music with heavy beats. Simultaneously estimating the BPM of two songs allows one or both of the songs to be time shifted or otherwise scaled to match the BPM of the songs so that they may be smoothly combined and played simultaneously, thereby creating a new mixed song that is a combination of both songs. Similarly, such conventional schemes allow the selection of an appropriate speed change and/or time shift to be applied to one or both songs so as to smoothly transition between two different pieces of music.
Most conventional mixing schemes focus simply on estimating the (BPM) of each song. In the simplest approach, a DJ simply changes the speed of the first and/or second song until the BPM's match, and then manually finds an offset in the songs to match up or align the beats. More sophisticated schemes use the computed BPM for each song to automatically determine an offset for alignment by automatically finding the locations of the beat sounds.
Unfortunately, such schemes tend to perform poorly where the BPM of one or more of the songs is not clearly discernable, or where the BPM varies or shifts over time. In such cases, conventional mixing schemes often fail to provide an alignment or mixing which maintains a reasonable quality across such changing BPM's. Any misalignment of the songs is then typically readily apparent to human listeners.
Some work has been done in estimating a beat structure of a single piece of music, rather than simply computing a BPM for that piece of music. Such schemes can also be used for aligning two or more pieces of music. For example, one such scheme estimates a beat structure via correlations across a number of filter banks. Another scheme provides a probabilistic approach that allows for variation in the beat of a song. Each of these methods are capable of estimating the beat structure of a song, however, if they were to be used to align two pieces of music, each would be susceptible to problems similar to the schemes which operate on simple BPM computations because they consider each song separately, and then estimate or compute time scaling and alignment in the same manner as the BPM schemes described above.
One problem common to all of the above-mentioned mixing schemes is an inability to successfully mix songs of significantly different genres. For example, the above-mentioned schemes are typically capable of mixing techno/dance songs (i.e., songs with significant beats and strong beat structure). However, these schemes will typically produce unacceptable results when attempting to mix songs of widely varying genres, such as, for example a Techno-type song having strong beats or beat-like sounds, with a piece of classical piano music that does not have strong beats.
Therefore, what is needed is a system and method for automatically aligning two or more songs for blending or mixing either all or part of those songs for at least partially simultaneous or overlapping playback (i.e., song transitioning or full mixing). However, because not all songs have strong beats, such a system and method should be able to mix in cases where one song has strong beats and the other does not without the need to actually determine the BPM of either song. Further, such a system and method should be computationally efficient so as to operate in at least real-time or faster.